BJSS data engineers are specialist software engineers that build, optimise and maintain data applications, systems and services. This role combines the discipline of software engineering with the knowledge and experience of building data solutions in order to deliver business value.
As a BJSS data engineer you’ll help our clients deploy data pipelines and processes in a production-safe manner, using the latest technologies and with a DataOps culture.
You’ll work in a fast moving, agile environment, within multi-disciplinary teams of highly skilled consultants, delivering modern data platforms into large organisations.
You can expect to get involved in variety of projects in the cloud (AWS, Azure, GCP), learning about and using data services such as Databricks, Data Factory, Synapse, Kafka, Redshift, Glue, Athena, BigQuery, S3, Cloud Data Fusion etc.
Requirements :
You're an engineer at heart and enjoy the challenge of building reliable, efficient data applications systems, services and platforms.
You have a good understanding of coding best practices and design patterns and experience with code and data versioning, dependency management, code quality and optimisation, error handling, logging, monitoring, validation and alerting.
You have experience in writing well tested object-oriented Python.
You have experience with using CI/CD tooling to analyse, build, test and deploy your code.
You have a good understanding of design choices for data storage and data processing, with a particular focus on cloud data services.
You have experience in using parallel computing to process large datasets and to optimise computationally intensive tasks.
You have experience in programmatically deploying, scheduling and monitoring components in a workflow.
You have experience in writing complex queries against relational and non-relational data stores.